Generate docs on install, nicer URLs and some caching #107

line-o · 2025-08-03T13:24:41Z

This PR is built on top of #106

The client-facing URLs are changing, legacy URLs are redirected to their new counterpart

the html part is dropped from all URLs
- /index.html -> /
- /view.html -> /view
- /ajax.html -> /query
- /browse.html -> /browse

the type parameter for queries was renamed to where to better indicate what the parameter does. It determines where the entered term is searched in; "everywhere" being the default.

Other enhancements

improve XQDoc generation (add arity, parameters, annotations, module variables, ...)
- new xqdoc structure is used to improve rendering (arity, parameters, ...)
- this will allow future improvements
function signatures are rendered server-side
generate documentation in the finish step of the package installation
add cache headers to most routes that could benefit from client-side caching
reliably set the URL on all rendered HTML pages
move all page templates to /templates/pages

- footer sticks to the bottom of the window - improve module info display - info icon size and alignment - module info boottom and right margin - module description foont size - improve function info display - remove duplicate function name

- rewrite scan.xql to generate.xqm - adhere to naming conventions - explicitly allow routes and deny evertying else - route modules/reindex.xql moved to /regenerate - add routes fro markdown and static resources - call to /regenerate is a GET request

- use new route `/exist/apps/fundocs/generate` - do not test for removed headline in function info - test for changed module name

- generate documentation in the finish step of the package installation - add cache headers to most routes that could benefit from client-side caching - the html part is dropped from all URLs - /index.html -> / - /view.html -> /view - /ajax.html -> /query - /browse.html -> /browse - reliably set the URL on all rendered HTML pages - move all page templates to /templates/pages

adamretter · 2025-08-06T12:34:28Z

The client-facing URLs are changing

Will the old URLs be maintained? Many people will have these bookmarked and they are likely also in printed documents including possibly the eXist-db Book.

As the W3C say - "Cool URIs don't change" - https://www.w3.org/Provider/Style/URI

- add arity - add annotations - add return with occurrence (return in comment left intact) - add parameters with occurrence (param elements in comment left intact)

line-o · 2025-08-17T12:40:42Z

The breaking nature is the reason I decided to open an extra PR.

User facing URLs that could have been bookmarked or printed:

/exist/apps/fundocs/index.html -> /exist/apps/fundocs/
/exist/apps/fundocs/view.html -> /exist/apps/fundocs/view
/exist/apps/fundocs/browse.html -> /exist/apps/fundocs/browse

Pro	Contra
There is not much to redirect	unnecessary cruft on installations that cannot have been bookmarked
It's the right thing to do™	Extra effort and maintenance
	Could also be handled by a proxy running in front of exist-db.org

adamretter · 2025-08-17T16:09:47Z

There are a LOT more URLs than that @line-o

line-o · 2025-08-17T16:23:11Z

Would you be willing to share an example URL I missed?

dizzzz · 2025-08-17T19:23:54Z

or we "chose them very badly" :-) anyway @adamretter it really helps to provide a few concrete samples in addition to the oneliner.

line-o · 2025-08-18T10:15:22Z

I just re-checked the controller and there is exactly one route matching *.html and there is exactly four existing files ending in html.
@adamretter Is it possible you mixed up documentation and function-documentation apps?

duncdrum

Looks good.

+1 for major version bump.

@line-o I think we could add custom forwards for the 4 old URLs ending in .html

- redirect /index.html to / with parameters q, action and map type parameter to where - redirect /browse.html to /browse with parameters w3c, extensions and appmodules - redirect /view.html to /view with parameters location, uri and function - add tests for redirects

The function documentation is now built when installing the package.

duncdrum

Great stuff, thanks @line-o . The tests are hardcoding the host, which will break when running with a different baseUrls. pathname takes globs and can be used instead of url to maintain test isolation. see cy.location

duncdrum · 2025-09-01T08:27:32Z

src/test/cypress/integration/redirect_spec.cy.js

+
+context('Visiting a legacy URL', () => {
+    it('redirects to the new index route', () => {
+      cy.visit('index.html?action=search&type=name&q=tei')


I would us qs object here and in the subsequent visits, for better legibility.

@duncdrum could you be more specific please. I am not familiar with all things Cypress.

duncdrum · 2025-09-01T08:29:07Z

src/test/cypress/integration/redirect_spec.cy.js

+context('Visiting a legacy URL', () => {
+    it('redirects to the new index route', () => {
+      cy.visit('index.html?action=search&type=name&q=tei')
+      cy.url().should('equal', 'http://localhost:8080/exist/apps/fundocs/?action=search&where=everywhere&q=tei')


cy.location('pathname') will let you test the redirects without needing to hardcode the host

The test also asserts the right querystring. Is that what qs is for from the earlier comment?

duncdrum · 2025-09-01T08:29:41Z

src/test/cypress/integration/regenerate_spec.cy.js

+  it('succeeds', () => {
+      cy.timeout(10000); // generation needs some time
+      cy.request({
+        url: 'http://127.0.0.1:8080/exist/apps/fundocs/regenerate',


use baseUrl fixed in the earlier PR

It's a pity that we have two divergent PRs now.

adamretter · 2025-09-01T09:16:54Z

@adamretter Is it possible you mixed up documentation and function-documentation apps?

@line-o No, I did not confuse them. Your change breaks all existing URI's that were book-markable and/or printable. These URIs have been stable for probably over a decade and appear in many places outside of the eXist-db project - including in print.

Three (of many many possible) examples:

There is no value in breaking these stable URIs! Please undo these changes.

dizzzz · 2025-09-01T18:16:01Z

I checked the URLs as mentioned, all work perfectly well, backwards compatible.

reinhapa · 2025-09-01T18:17:42Z

Three (of many many possible) examples:

https://www.exist-db.org/exist/apps/fundocs/browse.html?extensions=true

https://www.exist-db.org/exist/apps/fundocs/view.html?uri=http://exist-db.org/xquery/lucene&location=java:org.exist.xquery.modules.lucene.LuceneModule

https://www.exist-db.org/exist/apps/fundocs/view.html?uri=http://exist-db.org/xquery/lucene&location=java:org.exist.xquery.modules.lucene.LuceneModule#field.3

@adamretter after building the current content of this PR and deploying locally all the mentioned URLS (host part https://www.exist-db.org replaced with http://localhost did work flawlessly

adamretter · 2025-09-01T18:48:49Z

@reinhapa @dizzzz Sorry I don't understand, in the description of the PR by @line-o he writes:

The client-facing URLs are changing so this might be considered a breaking change.
the html part is dropped from all URLs

Above @line-o himself acknowledges that this is a breaking change!
If the .html part has been removed from the URLs, then that will break existing URLs!

Breaking such URLs after many many years is a very very bad idea!

dizzzz · 2025-09-01T20:26:46Z

Please avoid using "!" so frequently in your comments; I find these 'not so nice'.

The proof is in the pudding, 1st we did a code inspection and then we performed the actual testing where literally all URLs turned out to be functional and backwards compatible.

adamretter · 2025-09-01T23:03:57Z

the actual testing where literally all URLs turned out to be functional and backwards compatible.

@dizzzz Okay, thanks that's a relief!

Perhaps @line-o could update the description of the PR to reflect that the old URLs are still functional and there are no breaking changes?

line-o added 5 commits August 2, 2025 11:04

test: adapt tests to changed app

ff5ee89

- use new route `/exist/apps/fundocs/generate` - do not test for removed headline in function info - test for changed module name

fix: links in topnavigation bar

05d22d6

line-o requested review from duncdrum and a team August 3, 2025 19:01

line-o added 3 commits August 5, 2025 13:08

feat: improve function and module search

e3d5f00

chore: remove unused js code

3757698

chore: update to latest parent POM

c66b5af

line-o added 3 commits August 7, 2025 12:24

docs: add xqdoc schema file

b990af4

feat: generate better XQDoc

4f2aaf0

- add arity - add annotations - add return with occurrence (return in comment left intact) - add parameters with occurrence (param elements in comment left intact)

feat: improve function signature rendering

3f74ac1

duncdrum approved these changes Aug 25, 2025

View reviewed changes

dizzzz self-requested a review August 27, 2025 20:49

dizzzz approved these changes Aug 27, 2025

View reviewed changes

line-o added 3 commits August 30, 2025 10:33

test: remove unnecessary preparation for tests

507e2bb

The function documentation is now built when installing the package.

test: add test regenerating function documentation

5d59be9

duncdrum requested changes Sep 1, 2025

View reviewed changes

reinhapa approved these changes Sep 1, 2025

View reviewed changes

Uh oh!

Generate docs on install, nicer URLs and some caching #107

Are you sure you want to change the base?

Generate docs on install, nicer URLs and some caching #107

Conversation

line-o commented Aug 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adamretter commented Aug 6, 2025

Uh oh!

line-o commented Aug 17, 2025

Uh oh!

adamretter commented Aug 17, 2025

Uh oh!

line-o commented Aug 17, 2025

Uh oh!

dizzzz commented Aug 17, 2025

Uh oh!

line-o commented Aug 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

duncdrum left a comment

Choose a reason for hiding this comment

Uh oh!

duncdrum left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

duncdrum Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

line-o Sep 2, 2025

Choose a reason for hiding this comment

Uh oh!

duncdrum Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

line-o Sep 2, 2025

Choose a reason for hiding this comment

Uh oh!

duncdrum Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

line-o Sep 2, 2025

Choose a reason for hiding this comment

Uh oh!

adamretter commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dizzzz commented Sep 1, 2025

Uh oh!

reinhapa commented Sep 1, 2025

Uh oh!

adamretter commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dizzzz commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adamretter commented Sep 1, 2025

Uh oh!

Uh oh!

line-o commented Aug 3, 2025 •

edited

Loading

line-o commented Aug 18, 2025 •

edited

Loading

duncdrum left a comment •

edited

Loading

adamretter commented Sep 1, 2025 •

edited

Loading

adamretter commented Sep 1, 2025 •

edited

Loading

dizzzz commented Sep 1, 2025 •

edited

Loading